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Abstract 

Third grade students at a Midwestern elementary school participated in a 
9-week mobile learning intervention (MLI). Two classrooms used Everyday 
Math and daily practice using flashcards, etc., to learn multiplication. Two 
other classrooms used Everyday Math and web applications for the iPod 
touch for daily practice. MLI students outperformed comparison students 
on a postintervention multiplication test controlling for several covariates, 
including prior achievement, home iPod touch use, and previous teacher, 
among others. The medium-sized performance advantage (b = .217) was 
statistically significant at the .01 alpha level. The MLI influence on the most 
difficult multiplication items was also statistically significant but less impor¬ 
tant than the student’s demographic profile and the teacher’s advanced edu¬ 
cational technology degree. Experimental research is needed to assess longer- 
term achievement effects for diverse student groups and school settings and 
to explore how teaching and learning occurs in mobile environments above 
and beyond a particular device. (Keywords: mobile learning, mobile devices, 
student achievement, program evaluation) 


T he Who went mobile in 1971. Band members were in their mid-20s 
at the time. Today, a younger generation is “Goin Mobile,” albeit with 
iPhones, MP3 players, gaming devices, and tablet technology. Consid¬ 
er the following: 21% of upper elementary school students have a personal 
smartphone, a quarter of middle school students have a personal tablet de¬ 
vice, and more than half of high school students access the Internet outside 
of school via 3G/4G mobiles devices (Project Tomorrow, 2012). And, as The 
Who’s generation invested in and valued its mobility (via the automobile), 
the 21st century student similarly regards his/her mobile device (Wallace, 
2011). Traxler (2010) describes the “mobile attachment” this way: 

Almost every student owns one and uses one, often more than one. Not only 
do they own them and use them, but they also invest considerable time, ef¬ 
fort, and resources choosing, buying, customizing, and exploiting them. These 
devices express part or much of their owners’ values, affiliations, identity, and 
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individuality through their choice and use. They are both pervasive and 
ubiquitous, both conspicuous and unobtrusive, both noteworthy and taken 
for granted in the lives of most—but not all—students, (p. 149) 

Mobile Learning 

The prevalence of mobile devices among students (Brown & Green, 2010; 
Johnson, Adams, & Haywood, 2011) and access to the Internet and digital 
learning resources they provide (Eisele-Dyrli, 2011a; Park, 2011) is spurring 
schools to reconsider instructional and operational practices. Educators are 
leveraging this technology to improve student engagement, peer interaction, 
and collaboration; collect classroom feedback; improve communication; re¬ 
duce computer costs; and extend the place and time of learning (Allen, 2011; 
Kolb, 2011; Quillen, 2010; Shuler, 2009). Some are also using mobiles to 
enhance situated learning. For example, using mobile devices on field trips 
offers a window to socially construct learning through analytic tools and 
multimodal applications not tied to classroom spaces (Shuler, 2009). Still 
others, especially elementary teachers, are using skill development software 
and digital games to improve learning (Project Tomorrow, 2011). 

Mobile Learning Defined 

The conceptualization of mobile learning has evolved with the undergirding 
technology and resulting instructional possibilities. Traxler (2005) offered an 
early definition that specifically linked educational provision to a handheld 
device. Peng, Su, Chou, and Tsai (2009) situate the student within a context 
of ubiquitous and mobile computing technologies where relevant, conve¬ 
nient, expedient, and immediate learning is possible, while Park (2011) 
defines mobile learning as “the use of mobile or wireless devices for the 
purposes of learning while on the move” (p. 79). 

Wong (2012) offers a learner-centric conceptualization of mobile learn¬ 
ing, where continual knowledge construction occurs seamlessly along sever¬ 
al dimensions, including location (physical and digital, personal and social, 
informal and formal), time, pedagogy, and device type, among others. From 
this perspective, learners “are supposed to be knowledge builders who treat 
any material that they acquire from the Internet as resources to support their 
sense making and knowledge construction” (p. E21). Wong also discusses 
how a cloud-based “learner hub” may be coupled with a smartphone and a 
laptop computer to form a functional personal learning environment. 

Mobile Learning Examples 

Students consider the function of mobile technologies differently than their 
teachers and parents, and much of it is tied to productive learning. 

For todays students, emerging technologies such as mobile learning, on¬ 
line learning, and digital content hold great promise for creating a new 
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learning environment that not only engages them in contextually based 
content, but also enables greater personalization of the learning process 
and empowers them to explore knowledge with an unfettered type of 
curiosity that is too often missing from traditional classroom settings. 
(Blackboard K-12,2011, p. 3) 

Perhaps the most well-established mobile learning initiative is one- 
to-one computing, in which every student receives a laptop computer for 
school work. For example, the Irving Independent School District (http:// 
www.irvingisd.net/one2one) issues laptops to high school students to help 
transform instruction and improve learning. However, although positive 
results have been experienced by one-to-one implementers, uneven use and 
high equipment costs tend to diminish gains (Bebell & Kay, 2010; Bebell 
& O’Dwyer, 2010; Donovan, Green, & Hartley, 2010; Dunleavy, Dexter, 

& Heinecke, 2007; Goodwin, 2011; Greaves, 2010; Holcomb, 2009; Suhr, 
Hernandez, Grimes, & Warschauer, 2010; Weston & Bain, 2010). 

Using mobile devices for educational gaming is becoming popular in 
schools too. According to Shin, Norris, and Soloway (2011), “mobile gam¬ 
ing creates an individualized learning environment that allows students to 
select their own learning paths based on their prior knowledge and learning 
progress. This flexible approach, linked to prior knowledge, leads to mean¬ 
ingful learning” (p. 468). Oconomowoc High School (http://www.oasd.kl2. 
wi.us/page.cfm?p=5295) uses gaming to teach analytical, creative, logic, 
and problem-solving skills. Smartcodes downloaded to devices assist in 
board-game play, and augmented reality games are developed and played on 
mobiles. Research linking gaming to student achievement is emerging. For 
example, Young et al. (2012) reviewed more than 300 video gaming studies 
and found varying degrees of impact based on the academic subject (e.g., 
more for language and less for math). 

The “flipped” classroom takes advantage of students’ personal mobile 
technology and Internet access outside of school. In this model, students 
learn at home by watching lectures and collaborating with students and 
teachers via learning management systems and online resources such as 
Khan Academy (http://www.khanacademy.org). This homework approach 
frees classroom time for problem solving and project-based learning fa¬ 
cilitated by the teacher. Byron High School in Minnesota, USA, recently 
flipped its math classrooms to improve student achievement and save 
money on instructional materials by using a free online learning system 
(Fulton, 2012). 

Many schools are incorporating “iTechnology” (e.g., iPod, iPad) in learn¬ 
ing activities because of its versatility (e.g., e-reading, calculating, mapping, 
video and audio recording, Internet browsing, gaming), familiarity to stu¬ 
dents, and affordability. Banister (2010) also notes the educational potential 
of software apps designed specifically for these devices: 
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In the area of mathematics education, web applications for the iPod touch 
provide many ways for students to practice and explore computational 
problems. Flash Math is a basic drill program that presents the user with 
a set of timed problems. Settings for addition, subtraction, multiplication, 
and division, as well as choices for the number of place values to include 
in problems (up to at least 1,000,000,000), the number of problems to 
present, the time for each problem, and progressive time adjustments, 
make this application very customizable for the student. Teachers could 
certainly use this application to provide individual students with the types 
of arithmetic practice that they need to become more proficient in their 
mathematics classroom, (pp. 126-127) 

The Escondido Union School District’s iRead program (https://sites. 
google.com/a/eusd.org/eusd-iread) uses iPods in primary reading instruc¬ 
tion, and the Master’s Academy (http://www.mastersacademy.org/article. 
asp?articleID=75) issues iPads to high school students to “increase students’ 
capacity to be adaptable, to collaborate, to innovate, to create, to be produc¬ 
tive, and to engage technology in an ethical manner” (https://edseminars. 
apple.com/event/Uwl70-m0k4m). Similar to gaming research, the evidence 
of iTechnology impact on student achievement is limited, given its recency. 
iRead evaluators report more reading growth for its elementary iPod users 
compared with non-iPod users. Schneps, O’Keefe, Heffner-Wong, and Son- 
nert (2010) report a larger gain in content knowledge for students reading 
with an iPhone/iPod touch over conventional reading. 

Research Problem and Questions 

The need for empirical research connecting educational technology and student 
learning is increasingly cited in the literature (Bebell, O’Dwyer, Russell, & Hoff¬ 
man, 2010; Traxler, 2007). This need is underscored as mobile learning sprouts 
in PK-12 schools with little evidence to guide implementation decisions. The 
research that does exist usually describes mobile phone and PDA usage in higher 
education settings (Wu, Jim Wu, Chen, Kao, Lin, & Huang, in press). This study 
adds to the literature by examining the influence of a Midwestern elementary 
school’s Mobile Learning Intervention (MLI) on third grade math achievement. 
Specifically, a regression-adjusted covariate comparison group design (see Henry, 
2010) is used to address the following research questions: 

1. Does participation in the MLI explain a significant amount of variation 
on a postintervention multiplication test controlling for several covariates 
including prior student achievement? If so, what is the influence of the 
intervention relative to the control variables? 

2. Does participation in the MLI explain a significant amount of variation 
on the most difficult postintervention multiplication items controlling for 
several covariates, including prior student achievement? If so, what is the 
influence of the intervention relative to the control variables? 
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Background 

We implemented an MLI for third grade students at Park Elementary during 
the third quarter of the 2010-11 school year. Park is part of a Midwestern 
PK-12 school district that features a comprehensive high school, two inter¬ 
mediate schools, and four other elementary schools. The total student popu¬ 
lation (N = 5,000; White = 92%; male = 51%; economically disadvantaged 
= 20%; students with disabilities = 14%; English language learners =1%) 
reflects the demographics of the city and townships it serves. Educators and 
administrators are recognized as leading users of instructional and manage¬ 
ment technology. Recent initiatives include virtual schooling, game-based 
learning, wireless school networks, and iPads for students with disabilities. 

Park Elementary serves more than 500 students in grades PK to 4 (White 
= 88%; male = 54%; economically disadvantaged = 29%; students with dis¬ 
abilities = 16%; English language learners = 2%). There are 22 students in the 
average class. All classroom teachers are licensed, and half possess a master’s 
degree. The schoolwide student-to-computer ratio is 5:1. All classrooms are 
equipped with a computer, a document camera, and high-speed Internet. 
Technology shared among classrooms includes a mobile smartboard, digital 
Flip cameras, and a limited number of iPad and iPod devices. 

Schoolwide Math Curriculum 

Park students are taught math via the Everyday Mathematics (EM) approach 
(University of Chicago School Mathematics Project). Teachers use whole- 
group instruction to introduce concepts, followed by small-group or indi¬ 
vidual instruction. Teachers also communicate the “need to practice math 
skills” with parents and students via assignment notebooks and newsletters. 

Multiplication and division facts are introduced to third grade students 
during the second academic quarter, whereas more in-depth instruction 
occurs during the third quarter. Learning goals include automaticity with 
xO, xl, x2, x5, and xlO multiplication facts and using strategies to compute 
remaining facts up to 10 x 10. Teachers assess student learning via formative 
and summative assessments. Students also receive grades for math-related 
effort and attitude. 

Park’s Third Grade Mobile Learning Intervention 

Two intact classrooms (46 students, two teachers) used EM and daily prac¬ 
tice using “business as usual” techniques such as flash cards to learn multi¬ 
plication. Two other intact classrooms (41 students, two teachers) coupled 
EM and daily practice using iPod touch devices loaded with selected math 
apps. The total cost of intervention was $10,319. The iPod touch devices and 
sync cart accounted for 85% of the total cost. The remaining expense was 
related to instructional and technical support (see Table 1, p. 66). 

Park’s third grade teachers decided amongst themselves which two 
classrooms would implement the MLI. All participating students and 
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Table 1 . Mobile Learning Intervention: Total Cost and Per Student Cost 


Item 

Cost 

iPod touch device 

$5,280 ($220 X 24 
devices) 

Math apps“ 

$39 

Sync cart 6 

$3,475 

Technical support for startup 

$200 (8 hrs. x $25) 

Instructional support for startup 

$200 (8 hrs. x $25) 

6. Ongoing technical/instructional support 

$1,125(1 hr. per day x 
$25 x 45 days) 

Total cost of intervention 

$10,319 

Total cost of intervention per student 

$252 


3 Seven of 10 apps were “life” or free versions. 

b Components included the cart ($2,300), laptop ($1,000), and wireless access point ($175). 




Introduction of 
iPod touch devices 
and 10 math apps 
(one per day) 



Multiplication 

pretest 



Postintervention Multiplication Test 

controlling for: 

• Teacher master's degree in 
educational technology 

• Student demographic profile 

• Student absences during 
intervention 

• Student math effort/attitude during 
intervention 

• State third grade math test scale 
score before intervention 

• "Late" multiplication pretest 

• Previous teacher 

• Home iPod touch index 


Figure 1. Mobile Learning Intervention implementation timeline. 


teachers completed a pre-intervention survey to help identify and control 
for pre-existing group differences. We also collected a variety of other 
pre-intervention data (e.g., student test scores, report card data, atten¬ 
dance) from the school’s student information system for the same purpose. 
Classroom teachers administered a 50-item “late” multiplication pretest to 
all students after the iPod touch devices and math apps were introduced as 
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Table 2. Study Conditions: Mobile Learning Intervention vs. Comparison Group 


Mobile Learning Intervention Comparison 


Condition 

Classroom 1 

Classroom 2 

Classroom 3 

Classroom 4 

School 

Park 

Park 

Park 

Park 

Grade 

3 

3 

3 

3 

Classroom Teachers 

1 

1 

1 

1 

Classroom Technology 

PC, Internet, 

PC, Internet, 

PC, Internet, 

PC, Internet, 


document 

document 

document 

document 


camera 

camera 

camera, 

interactive 

whiteboard 

camera 

Math Curriculum 

Everyday 

Math 

Everyday 

Math 

Everyday 

Math 

Everyday Math 

Daily Math Instruction 

1-2 p.m. 

1-2 p.m. 

1-2 p.m. 

1-2 p.m. 

Daily Multiplication Practice w/ 
Rashcards, Fact Triangles, etc. 

No 

No 

Yes 

Yes 

Daily Multiplication Practice w/ iPod 
touch Device/Math Apps 

Yes 

Yes 

No 

No 

Mobile Synch/Operation Cart 

Yes 

Yes 

No 

No 

iPod touch Device 

Yes 

Yes 

No 

No 

Pre-Intervention Student Question¬ 
naire" 

Yes 

Yes 

Yes 

Yes 

Pre-Intervention Teacher Question¬ 
naire 6 

Yes 

Yes 

Yes 

Yes 

Multiplication Pretest (50 Items) 0 

Yes 

Yes 

Yes 

Yes 

Postintervention 

Yes 

Yes 

Yes 

Yes 


Multiplication Test (100 Items) 

3 The Pre-Intervention Student Questionnaire was given to all students to collect background information and to identify pre-existing 
group differences that may confound outcomes. 

b The Pre-Intervention Teacher Questionnaire was given to all teachers to collect background information and to identify pre-existing 
group differences that may confound outcomes. 

c The "Late” Pre-Intervention Multiplication Test was given several weeks into the intervention (after the MU students and teachers 
were introduced to the devices and math apps). Such a strategy is used to improve estimate precision (Schochet, 2010). 

another control variable to improve estimate precision (Schochet, 2010). 

We also administered a 100-item multiplication test to all students at the 
conclusion of the intervention. Figure 1 depicts the study timeline, and Table 
2 compares study conditions for the MLI and comparison groups. 

iPod touch Devices for Daily Practice 

The school district purchased 24 iPod touch devices for MLI students (see 
Figure 2, p. 68). Device features included 32 GB storage capacity, a built-in 
camera with HD video capability, a digital music player, a video player, messag¬ 
ing, an Internet browser, maps, calendars, a game center, a calculator, and an app 
store. Additionally, the district purchased a wireless mobile cart with an Airport 
Extreme wireless hub and laptop computer to store, charge and sync the devices. 
We numbered each iPod touch and assigned it to a specific student for daily 
practice. The device ID was displayed as wallpaper on the welcome screen. 
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Figure 2. iPod touch devices used in the Mobile Learning Intervention. 


Math Apps for Daily Practice 

Each iPod touch device contained the following math apps: Multiplica¬ 
tion Genius Lite (Blueonionsoft, Corp., 2009), Mad Math Lite (Lunch- 
box Apps, 2010), Pop Math (AppBlit LLC, 2009), Flash To Pass (Scybot 
Technologies, LLC, 2009), Math Drills Lite (Instant Interactive, 2010), 
Math Tappers: Multiples (HeavyLifters Network, 2010), Multiplication 
Flashcards To Go (box2 technologies LLC, 2011), Brain Thaw (Groovy 
Squared LLC, 2009), Math Magic (Anusen Inc., 2010), and FlowMath 
(palasoftware Inc., 2011). The apps were selected by the district’s in¬ 
structional technology administrator (ITA) and Park’s learning resource 
teacher (LRT), who are contributing researchers to this study, based on 
several criteria, including curriculum alignment, authentic skill prac¬ 
tice, operational ease, and attractiveness to students (e.g., game-like, 
touchscreen interaction, animated, multimodal, time challenges). The 
apps also reflected the standards-based multiplication skill practice and 
mastery expected of all Park third grade students, including logic and 
problem solving related to repeated addition and multiplication. Figure 3 
displays a sample of apps used in the MLI. 

Teacher Prep 

The ITA prepared a budget, contacted vendors, ordered equipment, acquired 
apps, and worked with District Technical Support to ready the mobile de¬ 
vices. The ITA also conducted meetings with the LRT and the four teachers 
to identify and reinforce multiplication topics to be covered during the third 
quarter. Meetings also focused on use of textbook and math practice materials, 
establishing that all students would practice 10 minutes each day (using iPod 
touch devices or traditional means) and that teaching of multiplication facts 
would be consistent across classrooms. 
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Multiplication Genius (Blueonionsoft Corp.) Math Tappers (HeavyLifters Network) 

Figure 3. Sample of iPod Touch math Apps used by Mobile Learning Intervention students. 


Participant Consent 

Student participation was voluntary. The ITA crafted and sent an informa¬ 
tional packet to parents. The parents indicated whether their child would 
participate and returned a signed consent form to the LRT. Parents were 
also informed that students would complete a pre-intervention survey that 
included questions about their home technology environment. The partici¬ 
pating teachers received a similar informational packet, consent form, and 
survey notice. 

Introducing the iPod touch Devices and Math Apps 

Park’s LRT was responsible for iPod touch and math app training, coordi¬ 
nating interclassroom iPod touch use, syncing and maintaining cart equip¬ 
ment, and serving as site liaison (i.e., communicating with teachers/parents/ 
administration). Although not part of the formal study, the LRT took field 
notes for her own reflection. The following excerpt provides insight about 
initial implementation and use of the mobile devices: 

The two classes that are using the iPod touches were introduced to the 
touches today. I reviewed “responsible use” using a doc cam, showed 
students the Pop Math app. I emphasized the goal of the app ... to earn 
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points by popping the factors and correct product, not just moving to the 
next level, since that can be done by just guessing. 

Students came up to the cart to retrieve their iPod touch. Once stu¬ 
dents had the iPod touches in their hands, I showed them how they 
could change the settings to “multiplication” and select a particular 
table that they needed to work on. Students spent ten minutes using 
the app and, there was no question about student engagement. I also 
noticed that students self-selected “tables” of numbers that they did 
not know well. 

I introduced an application a day for students to learn and use. A typical 
day looked like this: 

• I rolled the cart into the classroom. 

• I introduced one application to the students. 

• I did an overview of the application, using a document camera to 
project the screen of the iPod touch. 

• I always emphasized the goal of the application. 

• I showed how the application worked. 

• I showed how settings could be changed to work on specific tables. 

• Students retrieved their iPod touches from the cart and started work¬ 
ing on the application as soon as they got to their desks. 

• I walked around the room with the classroom teacher assisting stu¬ 
dents individually. 

• We would look at their progress to determine if they were working on 
tables that were perhaps too difficult or easy. 

• I helped students select the right table. 

• I helped students with strategies to learn/remember facts. 

• Students worked on the iPod touches for 10 minutes. 

Daily Math Practice in MLI and Comparison Classes 

MU classrooms. MLI students used the iPod touch devices and math apps 
to practice multiplication each day. Teachers often directed students to use 
one or two specific apps during the 10-minute period. On occasion, stu¬ 
dents were asked to focus on a specific multiplication table. For example, the 
teacher might ask students to work on 7s using a particular app. Other days, 
students self-selected an application. Students were never allowed to use 
more than two apps during a practice period. 
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Table 3. Pre-Intervention Student Demographics and Achievement: Mobile Learning Intervention vs. Comparison Group 



Mobile Learning Intervention (A/ = 41) 


Comparison (A/= 

46) 

Students 

% 

% 

X 2 

P 

Demographics 

Male 

56.1 

63.0 

.435 

.510 

Caucasian 

87.8 

95.7 

1.804 

.179 

Economically Disadvantaged 

24.4 

19.6 

.296 

.587 

Special Education 

12.2 

17.4 

.461 

.497 

Previous Teacher (Grade 2) a 

Teacher 1 

22.0 

28.3 

.457 

.499 

Teacher 2 

26.8 

28.3 

.022 

.881 

Teacher 3 

22.0 

13.0 

1.206 

.272 

Teacher 4 

22.0 

21.7 

.001 

.981 

Not Enrolled 

7.3 

8.7 

.056 

.813 


M 

M 

t 

P 

State Math Test Scale Score (Grade 3)" 

443.8 

440.0 

2.58 

.599 


Note. We collected this data from the school’s student information system. 
3 The student's second grade teacher. 
b The state math testis given each fall. 


The LRT worked with teachers to coordinate daily use of the mobile 
devices. On a typical day, one teacher used the devices after lunch recess. 
When students completed the 10-minute practice, the cart was rolled 
over to the other classroom, where students practiced after the formal 
math lesson. 

Comparison classrooms. Comparison students practiced multiplication for 
10 minutes each day using “business as usual” techniques (National Center 
on Response to Intervention, n.d.) such as flash cards, math games, fact 
triangles, and number sequences. Sometimes students chose their desired 
method of practice, and other times classroom teachers directed practice. 
One Comparison teacher provided students with a list of websites for home 
math practice. The teacher also allowed students to play EM games a few 
times in the computer lab. 


Method 


Participants 

Students. Eighty-seven students consented to participate in the study (MLI = 
41; Comparison = 46), representing 97% of the school’s third grade popula¬ 
tion. The groups appeared well matched on gender, race/ethnicity, economic 
status, special education status, prior year teacher, and performance on the 
state-mandated third grade math test (Table 3). 

Results from the pre-intervention survey indicated students’ home tech¬ 
nology environments were similar (Table 4). About two-thirds of students 
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Table 4. Students’ Home Technology: Mobile Learning Intervention vs. Comparison Group 


Home Technology 

Mobile Learning Intervention 
(N= 41) % 

Comparison (N = 46) % 

How many computers at home? 

3 or more 

46.3 

41.3 

2 

29.3 

28.3 

1 

24.4 

28.3 

0 

0.0 

2.2 

Do you have an iPod touch at home? 

Yes 

46.3 

52.2 

How often do you play with your iPod touch? 

Every day 

5.0 

13.0 

A few times a week 

25.0 

23.9 

A few times a month 

7.5 

4.3 

Hardly ever 

12.5 

21.7 

What do you usually do on your iPod touch? 

Play games 

30.0 

32.6 

Practice math 

5.0 

4.3 

Practice reading 

0.0 

2.2 

Go on the Internet 

2.5 

4.3 

Listen to music 

5.0 

10.9 

Watch videos 

5.0 

6.5 

What other technologies do you have at home? 

iPad 

12.5 

17.4 

iPod or MP3 player 

62.5 

71.7 

Nintendo 

70.0 

80.4 

Wii 

80.0 

73.9 

Xbox 

30.0 

32.6 


Note. We collected this data via the Pre-Intervention Student Questionnaire. 


reported having an iPod or MP3 player, and about half reported having an iPod 
touch. Gaming devices or systems were most prevalent in participants’ homes. 

Teachers. The four participating teachers, on average, shared similar 
teaching experience and teaching styles, according to pre-intervention sur¬ 
vey feedback (Table 5). The respondents also reported similar perceptions 
about how mobile devices could influence learning. For example, one 
MLI teacher wrote, “I think it engages students and helps them to practice 
skills in a fun way!” and a Comparison teacher noted, “I think the iPod 
touch will be very engaging for students. They are so comfortable with the 
technologies, similar to a cell phone, smartphone, video game—all things 
that they know well.” 
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Table 5. Teacher Characteristics and Technology Use: Mobile Learning Intervention vs. Comparison Group 

Mobile Learning Intervention (N = 2) Comparison (N = 2) 


Teacher 

Classroom 1 

Classroom 2 

Classroom 3 

Classroom 4 

Gender 

Female 

Female 

Female 

Male 

Teaching Experience 

21 yrs. 

14 yrs. 

12 yrs. 

19 yrs. 

Teaching Style 

Student-centered/ 

Student-centered/ 

Student-centered/ 

Student-centered/ 


differentiated 

differentiated 

differentiated 

differentiated 

Advanced Technology Degree 

- 

Master’s 

Master’s 

- 

Technology-Related Profes¬ 


Blogs, wikis, Jing 


Interactive 

sional Development: 




whiteboard 

Technology Expectations for 

Word processing, 

Websites for math skill 

Word processing, 

Word processing, 

Students: 

Internet for research 

practice 

Internet for research 

Internet for 


and math practice 


and math practice 

research 

Type of Technology 

Internet 

Internet 

Internet, interactive 

Internet 

Use in the Classroom 



whiteboard, iPods 


Frequency of Technology Use 

Rarely as whole 

Weekly 

Daily 

Daily 

in the Classroom 

class 




How Will the iPod touch 

Motivational tool 

Engage students 

Engage students 

Excellent tool for 

Improve Learning? 




students 

Home Technology Use 

Personal computer 
—used daily, and 
iPod—used four to 
five days a week 

1 have a laptop, Nook, 
Wii, DSI, iPod. We use 
these daily! 

Wireless laptops, 
smartphone, GPS, 
iPods—use it often 
every day 

Home computer 
daily, searching for 
information and 

enhanced 

communication 


Note. Data collected via the Pre-Intervention Teacher Questionnaire. 


In contrast, the Comparison teachers reported using technology more fre¬ 
quently in their classrooms, mostly for word processing and Internet research. 
One Comparison teacher explained, “I use technology with my students on 
a daily basis; this is more than in past years. This is my first year with a Smart 
board in my room and with iPods in my room. In the past I would use the 
Smart board for certain lessons in a unit; now I use it almost daily.” Converse¬ 
ly, one MLI teacher responded, “I only have one computer in my room, which 
makes it hard, because only one or two students can use it at a time.” 

Dependent and Control Variables 

Dependent variable. We constructed a 100-item postintervention multiplica¬ 
tion test (64 single-digit items, 36 double-digit items), and the ITA admin¬ 
istered it to students in a group setting under standardized conditions after 
reading a test administration script to the test-takers that said, for example, 
“You will have 10 minutes to answer as many problems as you can. Don’t 
worry if you are unable to finish the entire test. Just do your best work.” The 
ITA collected the tests, and we hand-scored them and entered the scores 
into SPSS for analysis. 

Control variables. We included several variables in the analytical model to 
help control for pre-existing group differences (Table 6, p. 74). We selected 
the variables, collected during the pre-intervention phase, for intervention 
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Table 6. Control Variables 


Variable 

Data Source 

1. Teacher Master’s Degree in Educational Technology 

(1 = Classroom 2 + Classroom 3,0 = Classroom 1 + Classroom 4) 

Pre-Intervention Teacher Questionnaire 

2. Student Demographic Profile 

(1 = White + male + regular education + regular lunch, 0 = not) 

Student Information System 

3. Student Absences During Intervention 
(Number of days) 

Student Report Card 

4. Student Math Effort/Attitude During Intervention 
(1 = Satisfactory or excellent, 0 = not) 

Student Report Card 

5. State Third Grade Math Test Before Intervention 
(Scale scores) 

Student Information System 

6. “Late” Multiplication Pretest 
(Number items answered correctly) 

Mobile Learning Intervention 

7. Previous Teacher (Grade 2) 

(1 = Teacher 3,0 = Teacher 1 + Teacher 2 + Teacher 4 + not enrolled) 

Student Information System 

8. Home iPod touch Index (0-8) 

iPod touch device (1 = iPod touch, 0 = not) 

iPod touch use (1 = everyday, 0 = not) 

iPod touch to play games (1 = play games, 0 = not) 

iPod touch to practice math (1 = practice math, 0 = not) 

iPod touch to practice reading (1 = practice reading, 0 = not) 

iPod touch for internet (1 = Internet, 0 = not) 

iPod touch for music (1 = listen to music, 0 = not) 

iPod touch for video (1 = watch video, 0 = not) 

Pre-Intervention Student Questionnaire 


Note. Independent variable: MU participation (1= Classroom 1 + Classroom 2,0=Classroom 3 + Classroom 4). Dependent variable: 
post-intervention multiplication test. 


relevance (e.g., prior student achievement, student demographics, absence 
during intervention) and sample-size considerations. Generally speaking, 
10-15 participants are recommended for each control variable (Shultz & 
Whitney, 2005; Song & Herman, 2010; Stevens, 1996). We administered a 
“late” pretest to students after MLI students were introduced to the mobile 
devices and math apps. The timed, paper-and-pencil 50-item multiplication 
test mirrored the posttest in form, content, and standardized administration. 

Data Analysis 

We used measures of central tendency to describe test performance. We 
used Ordinary Least Squares multivariate regression analysis to estimate 
MLI influence on postintervention test performance. We used standardized 
beta to identify the magnitude of MLI influence. We also calculated Cohens 
d to facilitate effect size comparison with other technology interventions. In 
this case, the unstandardized regression coefficient ( B ) was divided by the 
standard deviation of the dependent variable. 
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Table 7. The Influence of MU Participation on a Postintervention Multiplication Test: Total Items Answered Correctly 


Model 

B 

P 

b 

95% Cl for B 

Constant 

-29.152 

.084 



Mobile Learning Intervention 

7.259 

.002 

.217 

[2.83,11.68] 

Advanced Technology Degree 

4.705 

.037 

.141 

[0.29, 9.11] 

Demographic Profile 

2.205 

.331 

.065 

[-2.28, 6.69] 

Student Absences 

-.289 

.518 

-.042 

[-1.17, 0.59] 

Math Effort/Attitudes 

8.723 

.003 

.208 

[3.08,14.35] 

State Math Test 

.092 

.023 

.185 

[0.01,0.17] 

Late Pretest 

.899 

.000 

.588 

[0.64,1.15] 

Previous Teacher 

5.268 

.062 

.119 

[-0.27,10.80] 

Home iPod touch Index 

.985 

.280 

.070 

[-0.81,2.78] 

Adjusted R 2 

.681 




F 

21.371 

.000 




Note. N = 87. Cl = confidence interval. Adjusted mean predicted test scores for MU students (M= 54.5, SD= 14.8) and Comparison 
students (M= 46.3, SD= 12.5). d= 0.45,95% Cl [0.02,0.88]. 


Results 

Does participation in the MLI explain a significant amount of variation on 
the postintervention multiplication test? If so, what is the influence of the 
intervention relative to the control variables? 

On average, MLI students answered more items correctly on the postint¬ 
ervention multiplication test (M = 54.5, SD = 14.8) than the Comparison 
students (M = 46.3, SD = 12.5). We entered the independent and control 
variables in a single step for the regression analysis. Influence and multicol- 
linearity diagnostics were not significant. The analytical model explained 
68.1% of variance in the total number of multiplication items answered cor¬ 
rectly (Table 7). The medium-sized performance advantage for MLI students 
(b = .217) was statistically significant at the .01 alpha level. MLI participation 
was the most influential “explainer” of test performance excepting the pre¬ 
test. The student’s math-related effort/attitude grade was also an influential 
predictor of test performance (b = .208). 

Does participation in the MLI explain a significant amount of variation on the 
most difficult postintervention multiplication items? If so, what is the influ¬ 
ence of the intervention relative to the control variables? 

We repeated the analysis using double-digit multiplication items as the 
dependent variable. MLI students answered a greater number of these 
items correctly (M = 11.6, SD = 4.9) than the Comparison students (M 
= 8.2, SD = 4.4). Casewise diagnostics identified one significant outlier 
(one MLI student answered all items correctly), which we excluded from 
the regression analysis. The resulting model explained 42.9 % of variance in 
the outcome measure (Table 8, p. 76). MLI participation was a significant 
predictor of test performance but was less influential (b = .201) than several 

Volume 45 Number 1 | Journal of Research on Technology in Education | 75 
Copyright © 2012, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 



Kiger, Herro, & Prunty 


Table 8. The Influence of MU Participation on a Postintervention Multiplication Test: Double-Digit Items Answered Correctly 


Model 

B 

P 

b 

95% Cl for B 

Constant 

-8.611 

.317 



Mobile Learning Intervention 

2.542 

.028 

.201 

[0.28, 4.80] 

Advanced Technology Degree 

2.653 

.021 

.210 

[0.41,4.89] 

Demographic Profile 

2.754 

.019 

.213 

[0.47, 5.03] 

Student Absences 

-.158 

.486 

-.061 

[-0.60, 0.29] 

Math Effort/Attitudes 

.781 

.588 

.049 

[-2.07, 3.63] 

State Math Test 

.013 

.517 

.068 

[-0.02, 0.05] 

Late Pretest 

.285 

.000 

.481 

[0.15, 0.41] 

Previous Teacher 

2.709 

.060 

.163 

[-0.11,5.53] 

Home iPod touch Index 

.586 

.207 

.110 

[-0.33,1.50] 

Adjusted Ff 

.429 




F 

8.088 

.000 




Note. N = 86. Cl = confidence interval. Adjusted mean predicted test scores for MU students (M = 11.6, SD = 4.9) and Comparison 
students (M= 8.2, SD= 4.4). d= 0.30,95% Cl [-0.12,0.73]. 


control variables, including the student’s demographic profile (b = .213) and 
whether the classroom teacher had an advanced educational technology 
degree (b = .210). 


Discussion 

Students are “goin mobile” across the United States, and educators are adjust¬ 
ing instructional and operational practices to reap learning benefits. Most 
available research about mobile learning has focused on cell phone and 
personal digital assistant (PDA) use in higher education settings (Wu et al„ 
in press). Little research has been conducted in PK-12 settings or directed 
toward a particular academic subject area. The current study adds to the litera¬ 
ture by examining the influence of a Midwestern elementary school’s Mobile 
Learning Intervention (i.e., using iPod touch devices and selected apps for 
daily multiplication practice) on third grade math achievement. 

Key Findings 

MLI students outperformed Comparison students on a postintervention 
multiplication test controlling for prior student achievement and several 
other covariates. This finding suggests that coupling “business as usual” 
curriculum with a mobile device may be a cost-effective lever to improve 
student achievement. The costs and outcomes of the evaluand’s MLI also 
compare favorably to those of alternative programming. This is an impor¬ 
tant criterion for judging merit and worth (Harris, 2009; Hill, Bloom, Black, 
& Lipsey, 2007). For example, the MLI cost per student ($252) is fiscally 
preferably to one-to-one laptop programs and similar initiatives that require 
significantly more investment in hardware, software, and technical support. 
The MLI effect size (b = .217 or d = 0.45) also matches up with alterna- 
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tives like online learning, d = 0.24 (U.S. Department of Education, 2009); 
Computer-Managed Learning, Comprehensive Models, and Supplemental 
Computer-Assisted Instructional Technology, d = 0.17 (Cheung & Slavin, 
2011); and instructional technology in general, d = 0.41(Waxman, Lin, & 
Michko, 2003). 

We parsed MLI influence on the most difficult multiplication test items 
and found it to be statistically significant. Several control variables were also 
influentional explainers, including student demographics and advanced 
teacher training. This finding suggests that in-class mobile learning may 
foster and sustain productive student-teacher learning interactions. 

Implications for Practice 

The current study has several implications for practitioners. First, effec¬ 
tive implementation of mobile learning depends on administrative and 
school commitment and adequately trained teachers in matters of pedagogy, 
instructional technology integration, classroom management/facilitation, 
and mobile device operation. Second, an onsite resource person, like Park’s 
LRT, is needed for teacher support, device management and trouble shoot¬ 
ing, software/app selection, and stakeholder communication. The resource 
person may also facilitate a student’s out-of-school use of mobile devices and 
family involvement in related learning assignments. 

Adopting a bring your own device (BYOD) model may improve device 
accessibility while allowing the school to downsize its own hardware inven¬ 
tory. Similarly, sharing devices during an instructional or practice period 
may reduce hardware costs. Such a 1 :> 1 model can be advantageous from a 
pedagogical, collaborative, and productivity perspective (Larkin, 2011). And 
finally, a program evaluation plan should be formulated and executed for 
improvement and accountability purposes. 

Limitations of the Study 

We used a regression-adjusted covariate comparison group design to exam¬ 
ine the MLI influence on student achievement. As in all nonexperimental 
research, pretreatment group differences may confound posttreatment 
outcomes (Cook & Campbell, 1979). Pre-intervention data were collected 
to help identify and control for this threat. MLI participants were similar on 
measures of demographics, home technology use, enrollment history, and 
prior achievement. Teachers also appeared similar on most pre-intervention 
survey responses, including teaching experience and teaching style. In ad¬ 
dition, using a local comparison group and making statistical adjustments 
based on background variables, both used in the current study, reduce 
design bias (Glazerman, Levy, & Myers, 2003). 

We conducted the MLI over a relatively brief period under positive con¬ 
ditions. Such studies tend to produce better outcomes due to novelty effects 
and hyperattention to intervention details (Cheung & Slavin, 2011). For the 
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current study, novelty effects may be diminished, given the high percentage 
of participants reporting ownership of a personal iPod touch. In addition, a 
limited amount of iTechnology was available to classrooms for podcasting, 
etc. Park’s LRT may have affected study outcomes. This “second teacher” 
introduced math apps, provided technical support, and supported the in¬ 
tervention on an as-needed basis. However, it should be noted that instruc¬ 
tional support was minimal or absent beyond initial app training. 

And, whereas some may discount results of “best case” interventions, 
others argue that evaluation should begin within an optimal context before 
proceeding to a typical educational setting. As stated by Song and Herman 
( 2010 ): 

If an intervention has been found effective under ideal conditions, and 
the intervention itself has not changed substantially from what was tested 
in efficacy trial(s), the next logical step is to assess its impact under typical 
implementation conditions, (p. 362) 

Research consumers should also recognize that smaller studies (<250 par¬ 
ticipants) tend to yield larger and more variable effect size estimates. For ex¬ 
ample, Slavin and Smith (2009) report an overall negative correlation (-.28) 
between sample size and pretest adjusted effect size. As presented above, the 
current study compares favorably to alternative programming, and the effect 
size confidence interval for total multiplication items answered correctly did 
not contain the zero point suggesting a positive effect. 

Finally, this study did not, in effect, evaluate how learning occurred in the 
mobile environment. For example, the teacher’s role during mobile practice 
sessions was not a study focus. 

Recommendations for Future Research 

Experimental research at the classroom and school levels is required to 
confirm and extend results of the current study. Attention should be paid to 
the interaction of mobile learning and several covariates, including student 
demographics, student math-related effort/attitude, and teacher instruction¬ 
al technology background. Further cost analysis is needed to assist policy- 
related decisions and provide data for cost benchmarking (Harris, 2009). 

Additional study is also needed to evaluate how teaching and learning 
occurs within and across mobile environments. Wong’s (2012) multidimen¬ 
sional view of mobile learning may be a useful architecture to guide this 
inquiry. Possible research questions include: 

• Does mobile learning lend itself to a particular grade level, subject area, 
or instructional approach? 

• How might mobile learning affect assessment practices? 

• How might mobile learning affect traditional class schedules, student and 
teacher attendance requirements, and class size conception? 
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• What type of school network configuration and acceptable use policies 
best serve mobile learning? 

• How might schools partner with families and communities to facilitate 
learning outside the classroom via mobile technologies? 
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